Alternating dynamics is ubiquitous in real-world systems like, for example, speech, climatological data, physiological recordings (EEG/MEG), industrial processes or financial markets. Methods for the analysis of time-varying dynamical systems, which, aside from being non-stationary, might possibly be also nonlinear, are therefore an important issue for many application areas. In Kohlmorgen, 3., Müller, K.-R., Pawelzik, K. (1995). Improving short-term prediction with competing experts. In ICANN ,95: Proc. of the Int. Conf. on Artificial Neural Networks, EC2 & Cie, Paris, 2:215-220. and Pawelzik, K., Kohlmorgen, 3., Müller, K.-R. (1996). Annealed Competition of Experts for a Segmentation and Classification of Switching Dynamics. Neural Computation 8(2), 340-356. we introduced the annealed competition of experts (ACE) method for time series from non-linear switching dynamics, where an ensemble of neural network predictors specializes on different dynamical regimes by increasing the competition among the predictors through a deterministic annealing scheme. Related approaches for switching dynamics were presented in Bengio, Y., Frasconi, P. (1995). An Input Output IIMM Architecture. In: NIPS'94: Advances in Neural In Formation Processing Systems 7 (eds. G. Tesauro, D. S. Touretzky, T. K. Leen), Morgan Kaufmann, 427-434., Cacciatore, T. W., Nowlan, 5. J. (1994). Mixtures of Controllers for Jump Linear and Non-linear Plants. In NIPS, 93, (eds. J. D. Cowan, G. Tesauro, J. Alspector), Morgan Kaufmann, 719-726., Fancourt, C., Principe, J. C. (1996). A Neighborhood Map of Gompeting One Step Predictors for Piecewise Segmentation and Identification of Time Series. In ICNN, 96: Proc. of the Int. Conf. on Neural Networks, vol. 4, 1906-1911., Kehagias, A., Petridis, V. (1997). Time Series Segmentation using Predictive Modular Neural Networks. Neural Computation 9,1691-1710., Liehr, 5., Pawelzik, K., Kolilmorgen, J., Müller, K.-R. (1999). Hidden Markov Mixtures of Experts with an Application to EEG Recordings from Sleep. Theory in Biosciences 118, 246-260., Ramamurti, V., Ghosh, J. (1999). Structurally Adaptive Modular Networks for Non-Stationary Environments. IEEE Trans. Neural Networks 10(1), 152-60., Sbi, 5., Weigend, A. 5. (1997). Taking Time Seriously: Ilidden Markov Experts Applied to Financial Engineering. In CIFEr '97: Proc. of the Conf. on Computational Intelligence for Finanejal Engineering, IEEE, NJ, 244-252. For a brief review of some of these models, their advantages and drawbacks, see Frisch, K. R. (1955). The logarithmic potential method of convex programming. Memorandum, University Institute of Economics, Oslo., Husmeier, D. (2000). Learning Non-Stationary Gonditional Probability Distributions. Neural Networks 13, 287-290.
Further limitations and disadvantages of conventional, traditional, and proposed approaches will become apparent to one of skill in the art, through comparison of such systems with the present invention as set forth in the remainder of the present application.